System and method for contamination monitoring

ABSTRACT

A system for contamination monitoring includes a tracking component, a material identification component, a procedural component, and a notification component. The tracking component tracks an individual and one or more objects in a work area using a three-dimensional tracking system. The material identification component identifies a material of the one or more objects based on a captured image. The procedural component determines that an object of the one or more objects is contaminated based on tracked locations of the one or more objects and the individual. The notification component provides a notification of the contamination.

If an Application Data Sheet (ADS) has been filed on the filing date ofthis application, it is incorporated by reference herein. Anyapplications claimed on the ADS for priority under 35 U.S.C. §§119, 120,121, or 365(c), and any and all parent, grandparent, great-grandparent,etc. applications of such applications, are also incorporated byreference, including any priority claims made in those applications andany material incorporated by reference, to the extent such subjectmatter is not inconsistent herewith.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to and/or claims the benefit of theearliest available effective filing date(s) from the following listedapplication(s) (the “Priority Applications”), if any, listed below(e.g., claims earliest available priority dates for other thanprovisional patent applications or claims benefits under 35 USC §119(e)for provisional patent applications, for any and all parent,grandparent, great-grandparent, etc. applications of the PriorityApplication(s)). In addition, the present application is related to the“Related Applications,” if any, listed below.

PRIORITY APPLICATIONS

NONE

RELATED APPLICATIONS

-   -   U.S. patent application Ser. No. ______, entitled SYSTEM AND        METHOD FOR ACTIVITY MONITORING, naming Maxime Jean Jerome Bilet,        Roderick A. Hyde, Muriel Y. Ishikawa, Jordin T. Kare, Nathan P.        Myhrvold, Tony S. Pan, Nels R. Peterson, Clarence T. Tegreene,        Lowell L. Wood, Jr., Victoria Y. H. Wood, and Christopher        Charles Young as inventors, filed Feb. 26, 2013, with attorney        docket no. 46076/101, is related to the present application.

The United States Patent Office (USPTO) has published a notice to theeffect that the USPTO's computer programs require that patent applicantsreference both a serial number and indicate whether an application is acontinuation, continuation-in-part, or divisional of a parentapplication. Stephen G. Kunin, Benefit of Prior-Filed Application, USPTOOfficial Gazette Mar. 18, 2003. The USPTO further has provided forms forthe Application Data Sheet which allow automatic loading ofbibliographic data but which require identification of each applicationas a continuation, continuation-in-part, or divisional of a parentapplication. The present Applicant Entity (hereinafter “Applicant”) hasprovided above a specific reference to the application(s) from whichpriority is being claimed as recited by statute. Applicant understandsthat the statute is unambiguous in its specific reference language anddoes not require either a serial number or any characterization, such as“continuation” or “continuation-in-part,” for claiming priority to U.S.patent applications. Notwithstanding the foregoing, Applicantunderstands that the USPTO's computer programs have certain data entryrequirements, and hence Applicant has provided designation(s) of arelationship between the present application and its parentapplication(s) as set forth above and in any ADS filed in thisapplication, but expressly points out that such designation(s) are notto be construed in any way as any type of commentary and/or admission asto whether or not the present application contains any new matter inaddition to the matter of its parent application(s).

If the listings of applications provided above are inconsistent with thelistings provided via an ADS, it is the intent of the Applicant to claimpriority to each application that appears in the Priority Applicationssection of the ADS and to each application that appears in the PriorityApplications section of this application.

All subject matter of the Priority Applications and the RelatedApplications and of any and all parent, grandparent, great-grandparent,etc. applications of the Priority Applications and the RelatedApplications, including any priority claims, is incorporated herein byreference to the extent such subject matter is not inconsistentherewith.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic block diagram illustrating one embodiment of ancontamination monitoring system.

FIG. 2 is a perspective view illustrating one embodiment of a work areato be monitored.

FIG. 3 is a schematic block diagram illustrating one embodiment of anactivity monitoring component.

FIG. 4 is a plan view illustrating one embodiment of logical areas in awork area.

FIG. 5 is a perspective view illustrating an individual performing achopping activity in a work area, according to one embodiment.

FIG. 6 is an abstracted perspective view of the individual and work areaof FIG. 5, according to one embodiment.

FIG. 7 is a rear perspective view of an individual facing away from athree-dimensional tracking system, according to one embodiment.

FIG. 8 is a perspective view illustrating an individual performing acooking activity, according to one embodiment.

FIG. 9 is a perspective view illustrating an individual performing ahand washing activity, according to one embodiment.

FIG. 10 is a perspective view of a cut of meat, according to oneembodiment.

FIG. 11 is a perspective view of a group of carrots, according to oneembodiment.

FIG. 12 illustrates a flow chart of one embodiment of a method foractivity monitoring.

FIG. 13 illustrates a flow chart of one embodiment of a method forcontamination monitoring.

DETAILED DESCRIPTION

Food borne illness and disease can be dangerous and can cause serioussickness or death. In the case of contaminated restaurant food or massproduced food items, large numbers of people can be adversely affected.Generally, various food handling procedures, if followed, can reduce thechances of customers contracting these dangerous food borne illnessesand diseases. For example, simple procedures, such as hand washing,keeping raw meats or animal products separate from food items that areready to be consumed, refrigerating perishables at proper temperatures,and cooking foods to a sufficient heat, can significantly reduce therisk that food items served to customers might cause illness.

These various rules and methods are generally well understood andfollowed by highly skilled chefs or other food preparers. However, highstress situations, multiple workers, poor kitchen layouts, or othersituations can cause even the most experienced chefs to make mistakes.In the case of unskilled workers, failure to follow good kitchen rulescan be due to lack of knowledge, bad habits, or even laziness. Thus,kitchens with unskilled workers often implement extremely tightprocedures and processes such that the possibility for even unskilledworkers to make mistakes is significantly reduced. However, even tightlydesigned procedures cannot prevent all mistakes. Thus, even thoughprocedures to prevent food contamination are often known, lapses incomplying with these procedures still take place. Because of theseverity of potential consequences, such lapses in procedural complianceare not acceptable.

To encourage proper food handling, governments and employers oftencreate various laws and rules requiring compliance with a variety offood handling procedures. However, compliance with these laws and rulesare very difficult to enforce. In the case of legal requirements,compliance or non-compliance is generally determined based oninspections by government employees, tips from current or previousemployees, or complaints from the general public. However, such methodsof ensuring compliance often fail to capture how a kitchen or other foodhandling facility actually handles food and follows legal requirementson a day-to-day basis.

In view of the foregoing, Applicants have recognized that a need existsfor work area activity monitoring to detect violation of proceduralrules, contamination, or other potential problems. In one embodiment, asystem for contamination monitoring tracks an individual and one or moreobjects within a work area. The system identifies a material of the oneor more objects based on a captured image. The system determines whetheran object is contaminated based on the tracked locations of the objectsand the individual and provides a notification of a contamination.

The present disclosure provides examples and embodiments of activity andcontamination monitoring within a kitchen environment. However, kitchensare only one example of a work area that may benefit from activitymonitoring. Other work areas which may beneficially be monitored usingthe systems and methods herein include research laboratories, foodpackaging environments, food processing environments, product assemblyenvironments, manufacturing environments, healthcare environments, orthe like. For example, manufacturing or assembly facilities that processhigh-value items that could be damaged based on incorrect procedures,such as use of an incorrect tool, may violate regulations or damageproducts. Other example work areas to be monitored may include any workarea where transitioning of items from one activity or area to anothercan result in contamination, infection, or the like. One of skill in theart will recognize other environments that have important proceduralrequirements which may benefit from activity or contamination monitoringand which fall within the scope of the present disclosure.

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar components, unless contextdictates otherwise. The illustrative embodiments described in thedetailed description, drawings, and claims are not meant to be limiting.Other embodiments may be utilized, and other changes may be made,without departing from the spirit or scope of the subject matterpresented here.

FIG. 1 is a schematic diagram illustrating one embodiment of acontamination monitoring system 100. The contamination monitoring system100 may also be referred to herein as an activity monitoring system. Thecontamination monitoring system 100 may be used to monitor an individualand one or more objects within a work area. The contamination monitoringsystem 100 includes an activity monitoring component 102, athree-dimensional tracking system 104, a sensor 106, an output device108, and a remote computing system 110.

The activity monitoring component 102 monitors activity or contaminationwithin a work area. The monitoring may be used for training purposes,rule or legal compliance, food safety monitoring, or the like. Theactivity monitoring component 102 may monitor an individual and one ormore objects in a work area based on input from the three-dimensionaltracking system 104, sensor 106 and/or the remote computing system 110.The activity monitoring component 102 includes computer software and/orhardware to perform activity tracking, individual or objectidentification, and/or violation detection functions. The activitymonitoring component 102 may provide output to the output device 108,remote computing system 110, or other component or system to providenotification of the occurrence of a violation or contamination. Theactivity monitoring component 102 will be discussed in greater detail inrelation to FIG. 3.

A three-dimensional tracking system 104 obtains data about a work areaand any individuals or objects in the work area. The three-dimensionaltracking system 104 provides the data to the activity monitoringcomponent 102 for tracking the individuals and/or objects. In oneembodiment, the three-dimensional tracking system 104 includes a rangeimaging system. An example of a currently available range imaging systemthat may be used as three-dimensional tracking system 104 is theMicrosoft® Kinect®. Other systems that use projection and detectionranging methods may also be used. According to one embodiment, any rangeimaging system or body tracking system capable of determining a positionof an individual or one or more objects within a work area may be used.

The three-dimensional tracking system 104 may include one or moredevices for observing a work area and determining locations ofindividuals or objects in the work area. In one embodiment, thethree-dimensional tracking system 104 includes a range finder, such as alaser range finder, radar range finder, or other range finder fordetermining distances to objects or individuals in the work area.

The three-dimensional tracking system 104 may include one or moreimaging devices. In one embodiment, the three-dimensional trackingsystem 104 includes one or more visible wave length cameras. In anotherembodiment, the three-dimensional tracking system 104 includes one ormore infrared cameras. The cameras of the three-dimensional trackingsystem 104 may include still or video cameras. In one embodiment,three-dimensional information may be derived from stereo imaging devicesthat include two or more cameras having different points of view.

In one embodiment, the three-dimensional tracking system 104 includes arange finder and one or more cameras. The activity tracking system 100may include a plurality of three-dimensional tracking systems 104 forproviding more detailed coverage or to eliminate blind spots within thework area. Similarly, the cameras, range finders, or other devices of athree-dimensional tracking system 104 may be capable of panning tochange a field of view and/or zooming in or out to increase detail orbroaden a field of view. For example, the activity tracking system 100may provide system-controlled pan and zoom to provide intermittentcoverage to larger areas or detailed coverage to smaller areas.

The three-dimensional tracking system 104 may forward data obtained by acamera or range finder to the activity monitoring component 102 forprocessing. Alternatively, the three-dimensional tracking system 104 maybe configured to process the data and determine locations and/oridentify individuals and objects within the work area. Thethree-dimensional tracking system 104 may then forward the resultinglocation and/or identification information to the activity monitoringcomponent 102.

A sensor 106 may include a variety of sensor types for tracking activitywithin a work area. In one embodiment, a plurality of sensors 106 may beused to obtain various types of data within the work area. Examplesensor types include flow sensors, pressure sensors, identification tagsensors, temperature sensors, proximity sensor, and the like. Thesensors 106 may be fixedly mounted within the work area or may beintegrated into or mounted on an object or an individual. Theinformation gathered by the sensors 106 may be provided to the activitymonitoring component 102 for processing.

An output device 108 may provide output indicating that an activity hasviolated a rule, output that a contamination has occurred, or otherinformation regarding activity within the work area. The output device108 may include a display screen, an indicator light, a spotlight, alaser pointer, a speaker, or the like. In one embodiment, multipleoutput devices 108 may be included to provide notifications in a varietyof different ways within or outside of a work area.

The remote computing system 110 may receive violation or contaminationinformation from the activity monitoring component 102. The remotecomputing system 110 may be located at a remote location and may allow agovernment, company, or other organization to monitor a work area. Inone embodiment, the remote computing system 110 may receivenotifications regarding violating activities, food contaminations, legalviolations, or other procedural problems. For example, thethree-dimensional tracking system 104 and sensors 106 may be installedwithin a commercial kitchen and the activity monitoring component 102may report violations to the remote computing system 110 operated by agovernment, company headquarters, or the like.

The remote computing system 110 may also monitor operation of theactivity monitoring component 102. For example, the remote computingsystem may communicate periodically with the activity monitoringcomponent 102 to ensure that it remains operational. Similarly, theremote computing system 110 may provide software updates or otherinformation to the activity monitoring component 102.

In one embodiment, at least a portion of the contamination monitoringsystem 100 is transportable for use at different locations. For example,the activity monitoring component 102, three-dimensional tracking system104, sensor 106, and/or output device 108 may be transportable from onelocation to another to perform periodic inspections, periodic training,or the like. In one embodiment, temporary or permanent installation ofthe system 100 may be mandated in response to health violations or otherviolations.

The remote computing system 110 may also issue a citation based onprocedural problems. The citation may be provided to the output device108, or separately outside of the contamination monitoring system 100,such as via mail.

FIG. 2 is a perspective view of an example kitchen work area. Examplesof kitchen work areas may include commercial or non-commercial kitchens.The perspective view of FIG. 2 illustrates an example camera view thatmay be received by three-dimensional tracking system 104. The viewillustrates example work area objects, food items, and other objectsthat may be monitored by a contamination monitoring system 100. Thekitchen work area includes a plurality of counter surfaces 202. Thekitchen work area also includes a fridge 204 for storing food items oringredients, a washbasin 206 for washing dishes and tableware, a wastereceptacle 208, and the like. Also in the work area are a number of fooditems including a cut of meat 210 on a cutting board 212, a vegetable214, and ingredients 216 with a mixing spoon 218 in a mixing bowl 220.Plates 222 and knife set 224, with a plurality of knives, are alsoshown.

Additionally, portions of a contamination monitoring system 100 areshown. Face plates 226 of three-dimensional tracking systems 104 forobserving the kitchen work area are mounted within the kitchen area.Example output devices 108 that include a display 228, spotlights 230,and speakers 232 are also shown. A number of sensors (not shown) mayalso be located within the kitchen work area or the objects of thekitchen work area to provide input for tracking activities. A mirror 234may increase a field of view of a three-dimensional tracking system 104to allow monitoring of an individual who is facing away from athree-dimensional tracking system 104 and towards the mirror 234.

FIG. 3 is a schematic block diagram of an activity monitoring component102. The activity monitoring component 102 includes a tracking component302, an activity identification component 304, a material identificationcomponent 306, a procedural component 308, a notification component 310,a status component 312, an individual identification component 314, ahistory component 316, an activity storage component 318, a trackingstorage component 320, a violation storage component 322, a pestdetection component 324, a tracer detection component 326, and a tracertrigger component 328. The components 302-328 are given by way ofexample only and may not all be included in all embodiments. In fact,various embodiments may include only one or any combination of two ormore of the components 302-328 without limitation.

The activity monitoring component 102 may include one or more hardwareand/or software components for activity or contamination monitoring. Inone embodiment, the activity monitoring component 102 may include codeinstalled on a computing system and configured to be executed by aprocessor. In another embodiment, the components 302-328 of the activitymonitoring component 102 may be implemented across two or more devicesor computers. According to one embodiment, at least portions of one ormore of the components 302-328 may be included within athree-dimensional tracking system 104, sensor 106, output device 108, orremote computing system 110.

The tracking component 302 tracks an individual and one or more objectsin a work area. The tracking component 302 may track by storing orupdating a location of an individual or object. In one embodiment,previous locations or movements may also be detected, stored, and/orprovided to another module. In one embodiment, the tracking component302 uses a three-dimensional tracking system 104. For example, thetracking component 302 may receive video information or range findinginformation from a camera or range finder of the three-dimensionaltracking system 104. In one embodiment, the tracking component 302 mayreceive location information from the three-dimensional tracking system104 and use that information to track a current and/or previous locationof the individual. In another embodiment, the tracking component 302 mayreceive video footage information, image information, or range findinginformation and process the information to determine a location of theindividual or object. For example, the tracking component 302 mayreceive a video of the work area illustrated in FIG. 2 and process thevideo feed to track the location of one or more objects and anindividual.

The tracking component 302 tracks a location of an individual within awork area. For example, as an individual moves about the kitchen workarea of FIG. 2, the tracking component 302 may track the individual'slocation. In one embodiment, the tracking component 302 tracks thelocation of the individual in relation to one or more of the work areaobjects, such as the counter surfaces 202, a floor, a sink or washbasin206, a stovetop, an oven, a grill, a waste receptacle 208, a foodstorage location or container, or the like. For example, other types ofwork area objects may be encountered in types of work areas other thankitchens. In one embodiment, the tracking component 302 tracks thelocation of the individual in relation to logical areas in the workarea. The tracking component 302 may track specific portions of theindividual's body, such as where an individual puts his or her hands.

FIG. 4 illustrates a top view of a kitchen work area 400 that has beendivided into a plurality of logical areas. A hand washing area 402includes a sink and faucet for hand washing. A cold storage area 404includes fridges for storing perishable food items. A dish washing area406 includes a dishwasher, sinks, and counter space for washing dishes.A first foodstuff preparation area 408 includes counter space forpreparing food or food orders. A raw meat area 410 includes counterspace for preparing raw meats, raw animal products, or other food itemsthat need to be cooked prior to serving or have a danger of harboringfood borne illness or disease. A cooking area 412 includes a grill, astove, and an oven for cooking food. A storage area 414 includescabinets and drawers for storing food items, ingredients, and/or dishes.A foodstuff washing area 416 includes a sink and counter space to washfood items. A pass through 418 provides a sanitary counter space forpassing prepared plates or other food orders to servers, customers, orthe like. A second foodstuff preparation area 420 provides anothercounter space for preparing food items or food orders. An individual 422is also shown within the kitchen work area 400. The logical areas402-420 are examples only. Additional example areas which may be presentin some work areas include a foodstuff reception area, a supplyreception area, a sample reception area, an object unpacking area, atool washing area, an allergen free area, a sterile area, or the like.

According to one embodiment, as the individual 422 moves about thekitchen work area 400, the tracking component 302 tracks the location ofthe individual 422 within the logical areas 402-420. For example, thetracking component 302 may determine that the individual is within thefirst foodstuff preparation area 408 and may track the individual 422 asthe individual 422 moves to or through another logical area. Similarly,the location of one or more objects may be tracked as they are movedbetween different logical areas 402-420.

The tracking component 302 also tracks one or more objects within thework area. The one or more objects may include any objects or materialswithin the work area, such as those depicted in FIGS. 2 and 4. Someexamples of objects that may be tracked by the tracking componentinclude a food item, tableware, food preparation equipment, laboratoryequipment, laboratory samples, a tool, one or more parts to beassembled, a medical device, and a cleaning implement. Examples oftableware include items such as a utensil, a plate, a bowl, a cuttingboard, a knife, a mixing tool, a serving tool, and the like. One ofskill in the art will recognize that different types of work areas mayinclude different types of objects to be tracked.

The tracking component 302 may track a body position or gesture of theindividual 422. For example, the tracking component 302 may track thelocation and/or a position of an appendage of the individual 422. Thetracking component 302 tracks the location of the individual'sappendages, such as arms, one or both hands, head, and/or legs.According to one embodiment, each appendage may be tracked separatelyand/or independently. For example, the tracking component 302 may trackthe location of one hand within a first logical area and the location ofa second hand in a second logical area. Additionally or alternatively,the tracking component 302 may track a body position of the individual422. In one embodiment, tracking the body position comprises tracking aportion of the body with respect to another portion of the body. Forexample, the tracking component 302 may track a location of oneappendage in relation to another appendage or an angle formed betweenjoints. The tracking component 302 may also track movement of a hand ofthe individual or an object.

FIG. 5 illustrates a perspective view of an individual 422 within akitchen work area 400. An individual, and counter surfaces 202 aredisplayed with a variety of objects resting thereon. The individual islocated at one counter surface 202 chopping up a vegetable 214 on acutting board 212 using a knife 502. A cut of meat 210 is also locatedon the counter surface near the cutting board 212. On the cuttingsurface 202 behind the individual 422 a mixing spoon 218 in a mixingbowl 220 and a number of plates 222 are shown.

According to one embodiment, the tracking component 302 may track theindividual 422 within the work area 400. In one embodiment, the trackingcomponent 302 will track the movement of the individual's hands as theindividual cuts the vegetable 214 using the knife 502. The body positionof the individual 422 will be tracked. For example, the trackingcomponent 302 may track a location of the hands, arms, legs, back, head,and the like of the individual. Approximate angles between joints andrelative positions may also be tracked. In one embodiment, estimates ofpositions may be determined based on an anatomical model of theindividual or other software module. Thus, an anatomical or softwaremodel (such as used in the Microsoft® Kinect® may be used extrapolatemotions or activities not directly “visible”.

FIG. 6 illustrates an abstracted view of the work area 400 of FIG. 5. Anabstracted view of the individual 422, counter surfaces 202, cuttingboard 212, knife 502, carrot 214, cut of meat 210, plates 222, mixingspoon 218, and mixing bowl 220 is depicted. The abstracted view maydepict the type of information determined by a tracking component 302.For example, many details may be ignored by the tracking component 302.For example, each object may be treated uniquely and the location ofeach object may be tracked over time. Likewise, the exact appearance maynot be of importance although the location of the hands, head, and otherportions of the body are tracked to provide body position information.

According to one embodiment, the tracking component 302 may predict aposition of a first portion of the individual based on a position of asecond portion of an individual. For example, depending on the positionof the individual, a hand may be out of view of a three-dimensionaltracking system 104 while a shoulder or elbow may be within view. In oneembodiment, the tracking component 302 predicts the location of the handbased on the position of the shoulder or elbow. Similarly, the trackingcomponent 302 may predict a position of a first portion of theindividual's body based on movement of the first portion or movement ofa different portion of the body. In one embodiment, the trackingcomponent 302 predicts a body position of the individual based on ananatomical model. Using the anatomical model, the tracking component 302may be able to fill in unknown information based on assumptions from theanatomical model.

In one embodiment, predicting a body position may allow the trackingcomponent 302 to determine a location of an appendage of the individual,even if the appendage is in a blind spot of the three-dimensionaltracking system 104. According to one embodiment, one or more mirrors,or an additional three-dimensional tracking system 104 within the workarea, may be used to decrease blind spots.

FIG. 7 illustrates an example rear view of an individual who is facingaway from a three-dimensional tracking system 104. The individual'shands 702 are out of sight and are depicted with dotted lines. Accordingto one embodiment, because the individual's hands 702 are out of sight,or within a blind spot, of a three-dimensional tracking system 104, thethree-dimensional tracking system 104 may not be able to determine alocation of the hands 702 based on imaging or projection techniques.

However, an anatomical model may be used by the three-dimensionaltracking system 104 or tracking component 302 to predict a location ofthe hands 702 based on a position of the other parts of the individual422. For example, an anatomical model may include an estimate for thelength of the individual's forearms. More specifically, if thethree-dimensional tracking system 104 is able to locate an elbow and theangle formed by the angle, the locations of the hands may be predictedbased on the predicted distance between the elbow and the hand. Thus,even though the hands are out of sight, the three-dimensional trackingsystem 104 may be able to continue to track the location of theindividual's hands 702 and/or a body position of the individual.

In one embodiment, the tracking component 302 tracks two or moreindividuals within the work area. Tracking more than one individual mayallow the activity monitoring component 102 to identify problems thatare based on the actions of more than one individual.

In one embodiment, in addition to using a three-dimensional trackingsystem 104 to track the individual and objects, the tracking component302 uses an identification tag sensor to detect an identification tagand track a location. In one embodiment, the tracking component 302 maytrack the location of an individual or object based on detection of anidentification tag. For example, one or more identification tag sensorsdistributed throughout a work area may be used to determine anapproximate location of an identification tag and thereby track alocation of a corresponding individual or object. The identification tagsensor may detect a variety of types of identification tags such as aradio frequency identification (RFID) tag, a barcode, and/or a quickresponse (QR) code.

The tracking component 302 may, in some embodiments, track a location ofan individual or object based on output from other sensors. For example,output from sensors, such as pressure sensors, proximity sensors, orother sensors, may indicate a location of an individual or object andmay be used by the tracking component 302 to track a location of anindividual or an object.

Returning to FIG. 3, the activity identification component 304identifies an activity of the individual and any objects that areaffected by the activity. The activity identification component 304identifies an activity based on information received from one or more ofthe other components 302, 306-328 and/or from the three-dimensionaltracking system 104. For example, the activity identification component304 may identify an activity based on a location or a movement detectedby the tracking component 302 and/or the three-dimensional trackingsystem 104.

The activity identification component 304 may identify the activity ofan individual based on a location of the individual. For example, if anindividual is located in a cooking area, the activity identificationcomponent 304 may determine that the individual is cooking. Similarly,if an individual is located in a hand washing area, the activityidentification component 304 may determine that the individual iswashing his or her hands.

The activity identification component 304 may identify the activity ofan individual based on a proximity of the individual to an object. Forexample, if a user is proximal to a counter surface, a knife, and avegetable, the activity identification component 304 may identify theactivity as cutting the vegetable. Similarly, other activities may beidentified based on proximity to other objects.

The activity identification component 304 may identify the activity ofan individual based on a body position of the individual. In oneembodiment, the activity identification component 304 may receive bodyposition information from the tracking component 302 and identify theactivity based on the body position. The body position information mayinclude information regarding locations of the appendages of theindividual's body, angles of joints, and other information about thebody position. In one embodiment, the activity identification component304 may match a body position of an individual to a body positioncorresponding to an anatomical model of an individual involved in aspecific activity. The activity identification component 304 maydetermine that the individual is involved in the matching activity.

The activity identification component 304 may identify an activity basedon a gesture or movement made by the individual. As used herein, theterm “gesture” is given to mean movements or sequential positions of anindividual's body performed while engaged in an activity. For example, auser may perform a chopping gesture while cutting food that includes anup and down motion of a hand. A chopping gesture may also include astationary supporting hand for holding an object that is being chopped.Thus, in one embodiment, the activity identification component 304identifies a gesture that involves a stationary hand and an up and downmovement with another hand as chopping. Other example gestures mayinclude circular motions performed when mixing ingredients, rubbing twohands together while washing hands, or the like.

The activity identification component 304 may identify activities basedon one or more sensors in addition to, or alternatively to, using outputfrom a three-dimensional tracking system. Sensors may be located on orin one or more of an individual, a work area object, a food item, orother object. The activity identification component 304 may identify anactivity based on output from a flow sensor. For example, the activityidentification component 304 may identify an activity as hand washing,washing an object, or other washing activity in response to the flowsensor sensing the flow of water from a faucet. As another example, theactivity identification component 304 may identify an activity as handwashing, or washing an object in response to detecting a flow ofsanitizer from a dispenser.

The activity identification component 304 may identify an activity basedon output from a temperature sensor. In one embodiment, the temperaturesensor may be a sensor on an object whose temperature is to be sensed.In another embodiment, the temperature sensor may be a remotetemperature sensor to detect a temperature of an object or surface thatis not contacting the remote temperature sensor. For example, theactivity identification component 304 may identify the activity as acooking activity in response to detecting a high temperature.

The activity identification component 304 may identify an activity basedon output from a proximity sensor. For example, the activityidentification component 304 may identify an activity as using a workarea object based on the proximity sensor indicating that the individualis near the work area object. The activity identification component 304may identify an activity based on output from a pressure sensor. Forexample, the activity identification component 304 may identify that anactivity involves use of an object in response to the output from thepressure sensor indicating that the user is manipulating the object.Additionally, the pressure signature output by the pressure sensor maybe used by the activity identification component 304 to identify theactivity.

In one embodiment, the activity identification component 304 mayidentify an object involved in an activity based on an identificationtag sensor detecting an identification tag on the object. For example,the identification tag sensor may detect an RFID, barcode, or quickresponse code on a tag of an object and identify the object based on thedetected tag. The activity identification component 304 may thenidentify the activity based on the identified object. For example, ifthe identification tag sensor detects a tag corresponding to a knife asaffected by an activity, the activity identification component 304 mayidentify the activity as cutting using the knife, washing of the knife,or the like.

The activity identification component 304 may be configured to identifya wide variety of activities. Different embodiments may be configured toidentify an activity depending on a specific environment, such as foodpreparation, manufacturing, or healthcare. For example, some activitiesmay not occur or may not be of interest in some work areas. Some exampleactivities that can be identified by the activity identificationcomponent 304 are discussed below.

Returning to FIG. 5, one embodiment of a chopping activity isillustrated. In one embodiment, the activity identification component304 identifies the individual 422 as performing a chopping activitybased on the individual's location near counter surface 202, vegetable214, and cutting board, as well as based on movement of the knife 502and the individual's hand in an up and down motion (not shown).

FIG. 8 is a perspective view of an individual 422 involved in a cookingactivity. The individual 422 is shown holding a frying pan 802. In oneembodiment, the activity identification component 304 identifies theindividual 422 as involved in a cooking activity based on the individual422 holding the frying pan, the location of the individual 422 near astove, and/or output from a remote temperature sensor indicating thatthe frying pan 802 is at a cooking temperature.

FIG. 9 is a perspective view of an individual 422 involved in a handwashing activity. In one embodiment, the activity identificationcomponent 304 identifies the individual 422 as involved in a handwashing activity based on the individual's proximity to a sink, a flowsensor output indicating that water is flowing from the faucet 902, anda gesture of the individual 422 that includes rubbing the handstogether. The activity identification component 304 identifying anactivity as hand washing may also include determining that theindividual has used soap and rubbed the hands together vigorously for asufficient time period.

The activity identification component 304 may identify other types ofwashing activities. For example, the activity identification component304 may determine that an individual is washing an object based on theuser rubbing the object and/or applying a cleaning implement or cleaningsolution to the object. Similarly, the activity identification component304 may identify a user's activity as washing a food item. For example,the activity identification component 304 may determine that the user iswashing the food item based on the user holding the food item under arunning faucet.

The activity identification component 304 may identify a carryingactivity. For example, the activity identification component 304 mayidentify a carrying activity based on movement of an object with anindividual from one location to another. In one embodiment, the activityidentification component 304 may also determine that the individual isin contact or in sufficient proximity to contact the object.

The activity identification component 304 may identify a mixingactivity. For example, the activity identification component 304 mayidentify a mixing activity based on the individual holding a spoon orother mixing implement and moving the spoon in circular motions within abowl or other container.

The activity identification component 304 may identify an activity thatincludes contact between objects. According to one embodiment, theactivity identification component 304 identifies an activity thatincludes contact between individuals, objects, and/or work areasurfaces. The activity identification component 304 may identify contactbetween the individual and an object of the one or more objects. Theactivity identification component 304 may identify contact between theindividual and a surface of the work area, such as a counter surface.The activity identification component 304 may identify contact betweenthe individual and a floor, such as the hand of the individual and thefloor. The activity identification component 304 may identify contactbetween the individual and a sink. The activity identification component304 may identify contact between the individual and a waste product. Theactivity identification component 304 may identify contact between anobject and another object. The activity identification component 304 mayidentify contact between an object and a surface of the work area.

In one embodiment, the activity identification component 304 maydetermine that an object has contacted another object based on theobjects being within a threshold proximity. In one embodiment, theactivity identification component 304 may determine that an object hascontacted another object based on the two objects having correspondingmovements. For example, if an object is determined to be close to a handof an individual and the object substantially follows the movements ofthe hand, then the activity identification component 304 may determinethat the object is in contact with the individual. Additionally, if anobject accelerates or changes direction when it comes into proximitywith another object, the activity identification component 304 maydetermine that the two objects have contacted each other.

In one embodiment, the activity identification component 304 mayidentify a user's activity as using a tool. For example, the activityidentification component 304 may determine that the user is using thetool based on the user contacting the tool. The activity identificationcomponent 304 may also determine that the tool is being used on aspecific object based on the user manipulating the tool to contact theobject. The activity identification component 304 may identify a user'sactivity as assembling of parts. For example, the activityidentification component 304 may determine that the user is assemblingparts based on the user holding the parts and causing them to contacteach other.

According to one embodiment, the activity identification component 304may identify an activity as having a risk of contamination. For example,the activity identification component 304 may determine that an activityhas a risk of contaminating an individual, an object, or a portion ofthe work area. In one embodiment, the activity identification component304 may determine that anything that comes into contact with raw meat ora raw animal product is contaminated. In another embodiment, theactivity identification component 304 may determine that anything thatcomes into contact with unwashed hands is contaminated.

In one embodiment, the activity identification component 304 mayidentify an activity based on a previous or subsequent action. Forexample, the activity identification component 304 may identify anactivity differently based on an action or activity immediatelypreceding or immediately following the activity. Similarly, the activityidentification component 304 may identify the activity based on asequence of interactions between the individual and the one or moreobjects. For example, if a user grabs a knife, positions a cuttingboard, and places a material on the cutting board, the subsequent handmotion of the individual may be determined to be cutting the material,whereas similar detection motion of the individual holding a knifefollowing different actions may be determined to be washing the knife.

The material identification component 306 identifies a material withinthe work area. The material identification component 306 identifies thematerial based on information obtained or determined by thethree-dimensional tracking system 104, tracking component 302, or othercomponent.

In one embodiment, the material identification component 306 identifiesa material of an object or other material within the work area based ona captured image. For example, an image captured by a camera of thethree-dimensional tracking system 104 may be provided to the materialidentification component 306 for processing and identification ofmaterials. In one embodiment, a high-resolution camera may be used tocapture images of specific objects or materials to provide greaterdetail in allowing the material identification component 306 to identifya material. For example, a high-resolution or zooming still camera maybe used to obtain highly detailed pictures of objects or materials.Similarly, ranging information from a ranging system or information froma sensor 106 may also be used by the material identification component302 to identify a material.

The material identification component 306 may identify the materialbased on a variety of detected attributes of an object. Some exampleattributes that the material identification component 306 may use toidentify the material include attributes such as a color of thematerial, a pattern of the material, a shape of the material, a textureof the material, a size of the material or corresponding object, and thelike. The material identification component 306 may identify thematerial based on a color of the material. For example, the materialidentification component 306 may detect a color of a material in acaptured image and compare the color with a database to determine whattypes of materials have a similar color. The material identificationcomponent 306 may identify the material based on a pattern of thematerial. For example, the material identification component 306 maydetect a pattern in an image or in three-dimensional data provided bythe three-dimensional tracking system 104 and identify the materialbased on materials that may have the same or similar patterns.

The material identification component 306 may identify the materialbased on a texture of the individual. The texture of the material may bedetermined based on an image and/or three-dimensional data obtained bythe three-dimensional tracking system 104. The material identificationcomponent 306 may then identify the material based on the texturematching or being similar to a specific type of material. The materialidentification component 306 may identify the material based on a shapeof the material or object. For example, the shape of the material in thecaptured image or as determined by three-dimensional data from thethree-dimensional tracking system 104 may be used to match the materialor object with a material or object having a similar shape. In oneembodiment, the material identification component 306 may use one ormore of color, pattern, texture, shape, size, or the like to identifythe material.

Example materials that the material identification component 306 mayidentify include raw meat, cooked meat, animal products, baked goods,fruits, vegetables, or other food material or ingredients. In someembodiments, the material identification component 306 may also identifybodily fluids, chemicals, plants, allergens, metals, wood, or othermaterials. For example, in a health care environment it may be useful toidentify blood or other bodily fluids or products to determine sanitarystates of contacted surfaces.

FIGS. 10 and 11 illustrate example materials which may be identified bythe material identification component 306. FIG. 10 is a perspective viewof a cut of meat 1000. The cut of meat 1000 is shown with a marbledpattern. In one embodiment, the cut of meat 1000 includes various colorsof reds and whites. According to one embodiment, the materialidentification component 306 receives an image of the cut of meat 1000as it sits within a work area. The material identification component 306may then perform an image analysis of the cut of meat 1000 to determinethat it comprises raw meat, based on the color, pattern, and/or shape ofthe cut of meat. In one embodiment, three-dimensional data from thethree-dimensional tracking system 104 that indicates a three-dimensionalshape or texture of the cut of meat 1000 may also be used.

FIG. 11 is a perspective view of some carrots 1100. The carrots 1100have a bright orange color and an oblong shape. The carrots also have asurface pattern of ridges and bumps as well as greens at an end.According to one embodiment, the material identification component 306receives an image of the carrots 1100. The material identificationcomponent 306 identifies the carrots 1100 as carrots based on theircolor, shape, and texture.

According to one embodiment, the material identification component 306may identify a material as sanitary, unsanitary, or the like. Forexample, the material identification component 306 may identify materialas unsanitary if it has a high risk of contaminating another surfacethrough proximity or contact. In one embodiment, the materialidentification component 306 identifies raw meat as unsanitary due to arisk of transferring harmful bacteria to another surface throughcontact. Similarly, other raw animal products, bodily fluids, or othermaterials may also be identified as unsanitary.

The material identification component 306 may identify material ashaving a low risk of contaminating through proximity with an object ofthe one or more objects and the individual. For example, many fruits andvegetables have very low risks of harboring harmful bacteria, and thusmay be identified by the material identification component 306 assanitary. Similarly, baked goods, many plant products, or othermaterials may be identified by the material identification component 306as sanitary.

According to another embodiment, the material identification component306 may identify a material as an allergen or including an allergen. Forexample, the material identification component 306 may identify somenuts, flowers, leaves, or other materials as a specific type ofallergen. In one embodiment, specific allergens may be prohibited in awork area and the material identification component 306 may beconfigured to identify those allergens. For example, peanuts may beprohibited in a work area that is manufacturing peanut free foods.

The procedural component 308 determines whether a contamination orviolation of a procedural rule has occurred within a work area. Forexample, the procedural component 308 may identify errors such ashandling raw meat and then handling plates, putting raw meat on wrongcounters, or the like. In one embodiment, the procedural component 308determines whether an activity detected by the activity identificationcomponent 306 violates one or more procedural rules pertaining to one ormore of the individual, the work area, and the one or more objects. Inanother embodiment, the procedural component 308 determines that anobject of the one or more objects is contaminated based on trackedlocations of the one or more objects and the individual.

In the case of determining whether an activity violates a proceduralrule, the procedural component 308 compares the activity and/orcircumstances within a work area to a procedural rule. The proceduralrule may include one or more requirements in order to violate theprocedural rule. For example, a proximity requirement of a proceduralrule may require that objects and/or an individual are within a certainproximity from each other. The proximity requirement may be based on thelocations of objects, individuals, and/or logical areas within the workarea as tracked by the tracking component 302. Other requirements mayinclude sanitary requirements, location requirements, movementrequirements, body position requirements, or the like. One or moreprocedural rules may be stored by the procedural component 308. Avariety of example procedural rules are discussed below.

A procedural rule may define a violation of a food handling procedure, asanitary rule, or a contamination rule. For example, a contaminationrule may define work area situations or activities with potential tocontaminate a food item. Similarly, a procedural rule may definepotential contamination of a lab sample. The procedural component maycompare a work area activity, or circumstances within a work area, tothe procedural rule and determine whether a violation of a food handlingprocedure, a sanitary rule, and/or a contamination rule has occurred. Inone embodiment, a procedural rule may define proper kitchen conditions.For example, the procedural rule may include requirements for a maximumnumber of individuals, a temperature range, and/or locations for certaintypes of items within the kitchen work area.

A procedural rule may define a violation of proper tool usage, anassembly procedure, or the like. For example, a procedural rule for amanufacturing work area may include requirements that a specific tool isnot to be used during one or more assembly steps. For example, the toolmay provide excessive torque for the tightening of a certain type offastener. The procedural component 308 may determine whether the tool isbeing used and whether the fastener on which it is being used is of acertain type. If the conditions meet the requirements of the proceduralrule, the procedural component 308 may determine that a violation hasoccurred.

A procedural rule may define violation of a rule against the presence ofunauthorized individuals in a work area. The procedural rule may definethe presence of an unauthorized individual based on a determination thatthe individual is not wearing a badge. For example, the trackingcomponent 302 may track an individual within the work area and anidentification tag sensor may determine that the individual is notwearing a badge that includes an identification tag. Alternatively oradditionally, the procedural rule may define the presence of anunauthorized individual based on a determination that an individual'sface does not match a facial recognition map of any authorizedindividuals. For example, the tracking component 302 may obtain a facialrecognition map of an individual and the procedural component 308 maycompare the facial recognition map to recognition maps of authorizedindividuals stored within a storage device. If the facial recognitionmap does not have a match, the procedural component 308 determines thatthe individual is not authorized. Similarly, a procedural rule maydefine unauthorized performance of an activity. For example, onlycertain individuals may be allowed to operate certain machinery orperform certain activities with a work area.

A procedural rule may define a violation of a rule against theft. Therule against theft may define theft by an employee and/or theft by acustomer. For example, theft by an employee may be defined differentlythan theft by a customer because employees may be authorized to do morewith store merchandise without payment than a customer. For example, abartender within a bar is authorized to obtain beverages from behind acounter and pour drinks for customers while a customer is not authorizedto do so. However, a bartender may not be allowed to take payments fromcustomers and keep them as a tip. In one embodiment, the procedural rulemay define a violation of a rule against theft as a customer acceptingcash from a customer and failing to place it within a cash registerwithin a defined time period.

A procedural rule may define violation of a law. Examples of laws whichmay be violated include food handling regulations, criminal laws,packaging regulations, manufacturing regulations, or the like. In oneembodiment, the procedural rule may define requirements that constitutea violation of a law, and the procedural component 308 may compareactivities and/or circumstances within a work area to the definedrequirements.

A procedural rule may define a productivity rule. In one embodiment, aproductivity rule may include rules regarding late arrival, earlydeparture, activity in the work area, or the like. For example, theprocedural rule may include requirements that an individual be presentwithin the work area within a defined time period of the beginning ofthe individual's shift. The procedural component 308 may determinewhether the individual is present at the proper time, leaves early, orhas not physically moved enough for a certain time.

A procedural rule may define use of a forbidden substance. Example rulesagainst forbidden substances may include requirements indicating whethera substance is present in the work area or how a specific drug is used.For example, the material identification component 306 may identify aforbidden substance, and the procedural component may compare thepresence of the substance to a procedural rule to determine that thesubstance is forbidden. In one embodiment, labels on packaging may beused to identify a forbidden substance. As another example, a proceduralrule may define smoking, drinking alcohol, or using another drug, andthe procedural component 308 may detect whether an action thatcorresponds to the definition has occurred in the work area. Forexample, a procedural rule may define smoking as holding an elongatedand/or cylindrical object up to an individual's mouth.

In one embodiment, a procedural rule defines a combination of one ormore work area conditions that comprise a violation. The proceduralcomponent 308 may compare current conditions within a work area to theprocedural rule to determine whether the defined combination of workarea conditions is present. The work area conditions may include avariety of conditions as determined by the procedural component 308, orany of the other components 302-306 and 310-328 of the activitymonitoring component 102.

Examples of work area conditions which may be detected and compared to aprocedural rule include a sanitary state of an object, a sanitary stateof an individual, a sanitary state of a work area object or work areasurface, a sterile state of a work area object or surface, a proximitybetween objects, a location of an object in the work area, a currentactivity, a previous activity, conditions indicated by output from aremote temperature sensor, conditions indicated by output from aproximity sensor, conditions indicated by output from a pressure sensor,conditions indicated by output from an identification tag sensor,conditions indicated by output from a flow sensor, and the like.

In the case of determining whether an object is contaminated, theprocedural component 308 may analyze tracked locations of the object, aswell as tracked locations of other objects and/or an individual, todetermine whether the object has been contaminated. In one embodiment,the procedural component 308 determines whether an object iscontaminated based on the material of the object. For example, avegetable may be contaminated by coming in close proximity to raw meat,while a cut of raw meat may not necessarily be contaminated because itis close to some other raw meat.

In one embodiment, the procedural component 308 determines that anobject is contaminated based on contact with a contamination source. Thecontamination source includes, or has a high risk of including, amaterial that does not belong with the object. For example, acontamination source may have a high risk of having high bacteria levelsand high bacteria levels may not belong on food material that will beconsumed without cooking. The contamination source may include anotherobject, an individual, a work area object, or the like. For example,work area objects may include a counter surface, floor, sink, or otherwork area surface or object.

In one embodiment, the procedural component 308 will determine that anobject is contaminated based on the location of the object within alogical area of the work area. For example, the procedural component 308may determine that a counter surface in the logical area has a high riskof harboring high bacteria levels and that the object has contacted thecounter surface. The procedural component 308 may determine that anobject is contaminated based on the object's location within a meathandling area, a hand washing area, an allergen area, a non-sterilearea, or the like.

Some examples of contact with an object that the procedural component308 may determine to be contaminating are provided below. The proceduralcomponent 308 may determine that the object should be sterile butcontacted a non-sterile object. The procedural component 308 maydetermine that the object should be sanitary but contacted unwashedhands. The procedural component 308 may determine that the object shouldbe sanitary but has contacted raw meat. The procedural component 308 maydetermine that the object should be sanitary but has contacted rawanimal products. The procedural component 308 may determine that theobject should be allergen free but contacted an allergen. The object maybe a lab sample and the procedural component 308 may determine that thelab sample has contacted an object that has contacted another labsample. The procedural component 308 may determine that the objectshould be sterile but has contacted a material comprising bodily fluids.

According to one embodiment, the procedural component 308 determinesthat the object is contaminated based on determining that there is highrisk that the object has material that does not belong with the object.In one embodiment, the procedural component 308 determines that there isa high risk based on the location of the object in a logical area of thework area. For example, if a raw vegetable is carried into the raw meatarea 410 of FIG. 4, the procedural component 308 may determine that theraw vegetable is contaminated.

In one embodiment, the procedural component 308 determines that there isa high risk based on a proximity to a contamination source. For example,the object may be a plant product and the procedural component 308 maydetermine that there is a high risk of contamination because the plantproduct is proximal to a raw animal product. Turning to FIG. 5, avegetable 214 is being chopped by the individual 422 with a raw cut ofmeat 210 resting nearby. The procedural component 308 may determine thatthe vegetable 214 and the cut of meat 210 are too close and maydetermine that there is a high risk of contamination of the vegetable.Similarly, the procedural component 308 may determine that the knife502, cutting board, and counter surface 202 all have a high risk ofcontamination.

The procedural component 308 may determine that an object has a highrisk of contamination because it is proximal to a hand washing area. InFIG. 9, a cut of meat 210 is shown resting on a counter near a handwashing sink. In one embodiment, the procedural component 308 maydetermine that the cut of meat 210 is contaminated, or has a high riskof being contaminated, based on its proximity to the hand washing sink.Similarly, the procedural component 308 may determine that the cut ofmeat 210 is contaminated, or has a high risk of being contaminated,based on its proximity to the hand washing activity of the individual422.

The procedural component 308 may determine that an object that includesa lab sample has a high risk of contamination based on its proximity toanother lab sample. For example, lab samples used for research in alaboratory may be kept separate so that independent tests may beperformed. The procedural component 308 may determine that the labsamples are too close and may determine that one or both arecontaminated.

In one embodiment, the procedural component 308 determines that theobject is contaminated based on sequential contact between the object,one or more intermediate objects, and a contamination source. Forexample, even if a vegetable is not contacted or brought withinproximity of raw meat, the procedural component 308 may determine thatthe vegetable is contaminated based on contact with an object, hand, orsurface that has contacted the raw meat prior to contacting thevegetable.

One or more procedural rules may define scenarios where in increasedrisk of a violation or contamination is present, even if the violationor contamination has not yet occurred. For example, a procedural rulemay define when current activities present an increased probability of aproblem or violation. Thus, a recommendation or warning may be providedeven before a negative event occurs. For example, the proceduralcomponent 308 may detect the increased probability and cause anotification component 310 to provide a warning indicating of proximitybetween a contamination source and a sterile surface. The proceduralrules may define other types of hazards such as liquid splashing. Avoice notification may state that “you are washing X in this sink whileY is within 2 feet; be careful of splashing.” Similarly, the warning mayinclude a voice warning that states “you are carrying meat into thevegetable preparation area; bad idea!” The corresponding individual maythen be able to stop and prevent a violation that might otherwise haveoccurred.

The notification component 310 provides notification that indicates oneor more of a contamination, an occurrence of a violation, a status of anobject or individual, or the like. The notification component 310 mayreceive information regarding a contamination, violation, or status asdetermined by the procedural component 308 or other component and mayprovide a notification regarding the occurrence. The notificationcomponent 310 may provide notifications of contamination, violation, orstatus component in real-time.

According to one embodiment, the notification component 310 provides anotification by providing a data file that includes informationregarding a violation, contamination, status, or the like. In oneembodiment, the data file may be stored within memory. For example, thefile may include a time, individual involved, objects involved, type ofviolation, or other information, as determined by one or more of thecomponents 302-324 of the activity monitoring component 102. In oneembodiment, the file may be stored for later access by an employer,government, or other monitoring entity.

In one embodiment, a notification is provided to a remote location, suchas the remote computing system 110. For example, a remote computingsystem 110 may be used by an employer, government, or other organizationto determine whether a contamination or violation has occurred. A datafile may be provided to the remote location that provides informationabout the violation or contamination.

In one embodiment, the notification component 310 provides notificationwithin the work area. For example, the notification component 310provides a notification within the work area such that an individual 422working with the work area may be able to observe the notification andknow that the violation or contamination has occurred. The notificationcomponent 310 provides the notification in response to the proceduralcomponent 308 determining that a violation or contamination hasoccurred.

According to one embodiment, the notification component 310 may providea visible notification within the work area. In one embodiment, thenotification component 310 may cause a spotlight, laser pointer,indicator light, display, or other output device to provide a visibleoutput. In one embodiment, the notification component 310 may spotlightan object, individual, or surface that has been contaminated, asdetermined by the procedural component 308. For example, a spotlight,laser pointer, or the like may be used to point to an object that hasbeen contaminated. An individual may be able to discard, wash, orotherwise remedy the situation of the contaminated object. In oneembodiment, the notification component 310 may spotlight an object,individual, or surface that has been affected by a violating activity asdetermined by the procedural component 308.

In one embodiment, the notification component 310 may change a status ofan indicator light on an object to indicate the status of the object.For example, an object that may be used as a tool, cooking implement,cleaning implement, or the like may include an indicator light. Anindicator light may also be included in a badge, work area object, or atanother location to indicate a status of an individual, counter surface,or the like. The indicator light includes a small light emitting diode(LED) or other lamp. Circuitry embedded in the object may allow thenotification component 310 to wirelessly control the state of theindicator light. For example, the notification component 310 may providea wireless signal to the object to cause the state of the light tochange to indicate a contaminated status when the procedural component308 determines that the object is contaminated. In one embodiment,different colors may be used to indicate whether the object needs to becleaned, discarded, or the like. Similarly, a state of the lamp, such ason, off, or blinking, may indicate the status. As an example, thecutting board 212 of FIG. 5 may include an indicator light whose statemay be changed to a flashing red color to indicate that there has beenmeat on the cutting board or a plurality of other states.

In one embodiment, a display, such as the display 228 of FIG. 2, may beused to provide a visible notification of a contamination, violation, orother occurrence or status. For example, the notification component 310may display a diagram of the work area on the display 228. Locations ofobjects, individuals, work area objects, and/or logical areas may beindicated on the diagram. The notification component 310 may change acolor or other display attribute of an object, individual, or work areaobject to indicate contamination, a violating event, or other status.For example, if an individual performs an activity that results in thecontamination of a food item, the diagram may be updated to display theindividual and food item with a red color.

Using the spotlights, laser pointers, indicator lights, or diagrams, thenotification component 310 may be able to indicate safe, unsafe, orunknown spots on surfaces. Individuals within the work area may then beable to take actions with regard to different sources to reduce chancesof violations or contaminations.

According to one embodiment, the notification component 310 may providean audio notification within the work area. For example, thenotification component 310 may output an audio notification on thespeakers 232 of FIG. 2. In one embodiment, the notification component310 outputs a voice notification that indicates information regarding acontamination, violation, or status. For example, the voice notificationmay state a name of an individual, identify an object type and location,and state a procedural rule that has been violated. Similarly, the voicemay state a location in the work area, material, type, status, or otherinformation to provide a notification of the violation, contamination,or status. The voice notification may also state a rule or properprocedure to be followed in the future. An individual may be able torespond or learn from the event and remedy the current situation. Forexample, FIG. 9 illustrates a individual 422 washing hands near a cut ofmeat 210. The notification component 310 may provide a voicenotification stating that the cut of meat 210 is too close to the handwashing location and is in danger of being contaminated.

According to one embodiment, the notification component 310automatically issues a citation in response to the procedural componentdetermining that there was a procedural violation or contamination. Forexample, the citation may indicate the violation of a health regulation,manufacturing regulation, contamination of a lab sample, violation of atraining rule, or other error within the work area. The citation mayinclude information regarding a specific procedural rule, evidence ofthe violation as gathered by the contamination monitoring system 100,and/or a fine to be paid by an individual or establishment. In oneembodiment, the citation may be issued by a governing authority such asa government. According to one embodiment, the notification component310 and other components may work together to automatically detect,document, and issue citations for health violations or other violations.

In another embodiment, the notification component 310 provides anotification by issuing a report. The report may include a number ofviolations at an establishment within a time period. The report mayinclude a list of violations with descriptions and/or severity for eachviolation. In one embodiment, the report may include the number ofviolations of an individual within a time period. For example, anindividual may be identified as a “bad actor” or the cause of adisproportionate share of problems in the work area. A report may alsoinclude statistical information on compliance or noncompliance withspecific procedures or problems associated with particular individualsor particular activities (e.g., one individual may have a tendency toperforming hand washing in food sinks)

The status component 312 maintains a status of an object, a portion of awork area, an individual, or the like. The status maintained by thestatus component 312 may include a current status based ondeterminations made by the tracking component 302, the activityidentification component 304, the material identification component 306,the procedural component 308, or another component. For example, thestatus component 312 may update a status for an object in response tothe procedural component 308 determining that the object iscontaminated. The status maintained by the status component 312 mayinclude information regarding whether an object, individual, or workarea surface is contaminated, sterile, allergen free, or the like.Similarly, the status component 312 may update a status for an object inresponse to the tracking component 302 detecting a movement of theobject to a new location within a work area.

In one embodiment, the status component 312 maintains a database with anentry for an object, individual, and/or work area surface tracked by thetracking component 302. For example, an entry for the cut of meat 210,cutting board 212, vegetable 214, and other objects may be includedwithin a database. The status component 312 may update a status entrywhen a corresponding activity, movement, or other event occurs withinthe work area.

A status for an object, individual, or work area surface maintained by astatus component 312 may include a safe, unsafe, or unknown status. Forexample, the status component 312 may maintain a status that indicatesthe safety for consumption or contact with something that will beconsumed by a human. The safe status may indicate that there is littlerisk of danger from eating or handling food using an object. The unsafestatus may indicate that there is a known high danger for eating anobject or for eating off of an object, such as a plate or fork. Theunknown status may indicate that not enough information about the objectis known. In one embodiment, the activity monitoring component 102 maytreat an unknown status similar to an unsafe status. In anotherembodiment, the status component 312 may maintain a status that includesa contaminated, not contaminated, and unknown status for each trackedobject, individual, or the like.

The status component 312 may update a status based on an activityidentified by the activity identification component 304. The activityidentification component 304 may identify an activity that comprisescleaning an object, surface, or individual, and the status component 312may update a corresponding contamination status to indicate that theobject, surface, or individual has been cleaned. For example, the statuscomponent 312 may update a status to a safe or non-contaminated statusin response to the cleaning. The activity identification component 304may identify an activity that includes interaction between a firstobject and a second object, and the status component 312 may update acontamination status of the second object based on the contaminationstatus of the first object. Similar status updates may be made withrespect to interaction between individuals, objects, and/or portions ofa work area. In one embodiment, the status component 312 updates thecontamination status of an object, individual, or portion of a work areabased on the procedural component 308 determining that an activityviolates a procedural rule that defines violation of a contaminationrule.

According to one embodiment, the notification component 310 indicatesthe contamination status of an object, individual, or portion of thework area. For example, the notification component 310 may spotlight anyobject, individual, or portion of a work area that is contaminatedand/or needs to be cleaned. As another example, a diagram on a displaymay be updated to indicate the status of each tracked object, portion ofthe work area, and/or individual in the work area.

In one embodiment, a status component 312 maintains a tracking priorityfor each tracked object, individual, or portion of the work area. Forexample, some tracked entities within the work area may be of moreimportance than others. For example, because individuals generally domost of the moving and activities within a work area, they may have ahigher tracking priority than a work area surface that does not move.Similarly, objects or individuals that are contaminated or in an unsafestate may need to be tracked more closely to ensure that other objects,or portions of the work area are not contaminated. In one embodiment, atracking component 302 may receive the tracking priority and may trackan individual, object, or portion of the work area with a highertracking priority more accurately than an object, individual, or portionof the work area that has a lower tracking priority. For example, thetracking component 302 may more frequently check a location of an objectwith a higher tracking priority.

As another example, the tracking component 302 may more preciselydetermine a location in space of an object with a higher trackingpriority.

In one embodiment, the status component 312 may provide a status of anobject, individual, or portion of the work area to the proceduralcomponent 308. The procedural component 308 may use the statusinformation to determine whether a contamination or violation hasoccurred. In one embodiment, the procedural component 308 may includethe status component 312.

The individual identification component 314 identifies an individual inthe work area. The individual identification component 314 may identifyan individual based on physical attributes, facial recognition, or basedon an identification tag or badge. For example, the individualidentification component 314 may identify an individual based on imagescaptured by a camera, such as a camera of a three-dimensional trackingsystem 104. The individual identification component 314 may analyze theimage to determine one or more facial features and use those facialfeatures to substantially uniquely identify the individual. Theindividual identification component 314 may also determine one or moreother physical characteristics, such as height, proportional bodydimensions, or other characteristics to identify an individual. In oneembodiment, three-dimensional features obtained by a range findingdevice or other tracking system may also be used to identify anindividual. In another embodiment, an identification tag sensor may beused to detect an identification tag on an individual and therebyidentify the individual.

In one embodiment, the individual identification component 314 mayidentify each individual that enters a work area. For example, when aworker or other individual enters the work area the individualidentification component 314 may determine one or more attributes of theindividual. The individual identification component 314 may compare theattributes to one or more stored attributes to determine the identity ofthe individual. If there is a matching individual, the individualidentification component 314 determines the identity of the individualto be the same as the matching individual. If there is no matchingindividual, the individual identification component 314 may create a newentry with the determined attributes.

In another embodiment, the individual identification component 314 mayidentify an individual once a violation or contamination has occurred.Individuals that cause a problem or error may be identified and identityinformation may be logged. For example, the individual identificationcomponent 314 may determine attributes of an individual, store thoseattributes, and/or attempt to locate a corresponding entry for thatindividual in response to determining that a violation or contaminationcaused by the individual has occurred.

The history component 316 maintains a history of the work area. Thehistory component 316 may maintain a history that includes informationregarding past locations, contact between tracked objects orindividuals, activities or other information. For example, the historycomponent 316 may store location, status, cleaning, temperature,material type, identification information, or the like for each object,individual, and/or portion of a work area. In one embodiment, a time ofa washing or cleaning of an object, individual, or surface may be storedwithin the history. In one embodiment, the history may includeinformation about contact between objects and/or individuals.

According to one embodiment, the history component 316 maintains ahistory which allows reconstruction of what has occurred within the workarea. In one embodiment, one or more of the other components may be ableto access the history for their respective functions. For example, theprocedural component may analyze the history to determine whether aviolation or contamination has occurred. The status component 312 maydetermine a status of an object based on tracked locations and thetracked locations of other objects and/or individuals within the workarea.

The activity storage component 318 maintains a database of activities ofan individual. For example, all past activities identified by theactivity identification component 304 may be stored by the activitystorage component. Activities of one or more additional individuals mayalso be maintained within the database. In one embodiment, theprocedural component 308 determines whether the activity violates aprocedural rule of the one or more procedural rules based on one or moreof the activities of a single individual or of multiple individuals asmaintained within the database of activities. In one embodiment, theactivity storage component 318 stores information indicating theindividual who performed an activity. The activity storage component 318may analyze the database to determine a productivity of a specificindividual. For example, the activity storage component 318 maydetermine whether the individual is performing a sufficient amount ofactivities within a work shift.

The tracking storage component 320 stores information regarding trackingof the individual and the one or more objects in the work area. Forexample, the tracking storage component 320 may store information thatindicates the location of an object or individual within a work area.The stored tracking information may be used to prove the occurrence ofcertain events such as contaminations or violations of procedural rules.The tracking storage component 320 may store the tracking information inlocal storage or may provide the tracking information to a remotelocation for storage, such as the remote computing system 110 of FIG. 1.

According to one embodiment, the tracking storage component 320 storesvideo footage of a work area. For example, a video feed obtained by avideo camera of a three-dimensional tracking system 104 may be stored bythe tracking storage component 320. More than one video feed may bestored, for example, if multiple cameras are used. The video footage mayinclude video footage of an activity of an individual and/or thelocation of one or more objects in the work area.

In one embodiment, the tracking storage component 320 stores all videofeeds in a temporary storage buffer. After a buffer time, the trackingstorage component 320 may delete portions of the video older than abuffer time that do not correspond to a violation or contamination. Forexample, the tracking storage component may delete portions of videofootage older than two hours that do not correspond to a violation.Deletion of video footage may be desirable to decrease storagerequirements and/or to reduce privacy concerns. The tracking storagecomponent 320 may receive instructions from a procedural component 308indicating what time periods are relevant to proving the occurrence of aviolation. For example, if multiple activities lead up to and contributetowards a violation, the procedural component may provide a begin andend time for which video should be maintained.

In one embodiment, the tracking storage component 320 may transfer videofootage corresponding to a violation to long term storage prior todeletion from a buffer. The video footage corresponding to the violationmay include video footage of activities illustrating the violation ofthe one or more procedural rules. For example, the video footagecorresponding to the violation comprising video footage immediatelypreceding the violation and video footage immediately following theviolation.

According to one embodiment, the tracking storage component 320 storestracking information that includes abstracted information regarding oneor more of the work area, the activity of the individual, and the one ormore objects. The abstracted information may include information thatabstractly indicates the location, position, or orientation of theindividual or one or more objects. For example, the abstractedinformation may include position information that may be used toreconstruct body position or location information that may be used toreconstruct object location, but may not be include imagery or besufficient to reproduce a photo realistic image of the work area. Theabstracted information may include information regarding one or moreidentified activities. The abstracted information may includeinformation regarding a history of contact between the one or moreobjects, the individual, and portions of the work area. The abstractedinformation may be maintained within memory for a longer period of timethan video because it may be less memory intensive and/or may includedata that is less important for privacy reasons.

In one embodiment, the abstracted information may be used to reconstructthe information object location and information, as indicated in FIG. 6.For example, the abstracted information indicating the location of eachobject, a location of the individual, and a body position of theindividual over time may be used to replay events within the work areain an abstracted form. According to one embodiment, the abstractedversion of activities and/or locations within the work area may reduceprivacy concerns that may arise when video or imagery of individuals isstored and/or displayed at a later date.

The violation storage component 322 maintains a violation database ofviolating activities and one or more work area conditions associatedwith each violating activity. The work area conditions may includeinformation regarding output from sensors, a number of people within thework area, a location of the violation within the work area, objectsinvolved in the violation, the time of day, or the like. For example,each time a violating activity is detected, the violation storagecomponent 322 may add an entry indicating occurrence of the violatingactivity as well as work area conditions associated with the violation.In one embodiment, all available types of data obtained by sensors andother data regarding the work area at or around the time of theviolation is saved and associated with the violation.

According to one embodiment, the violation storage component 322analyzes the violation database to identify trends or correlations thatmay indicate problems which may be resolved to improve compliance withprocedural rules. In one embodiment, the violation storage component 322analyzes the violation database to identify a combination of theassociated workplace conditions having a correlation to a violatingactivity. For example, the violation storage component 322 may be ableto determine that a specific error occurs often around the same time ofday and when there are more than a specific number of individuals.Efforts to take a break at that time and/or reduce the number ofindividuals in the kitchen may help reduce occurrence of violations. Inone embodiment, the violation storage component 322 analyzes theviolation database to identify a procedure having a high correlation toa violating activity. For example, the violation storage component 322may identify that individuals often make errors when preparing aspecific dish. The procedure in place for the dish can then be reviewedand modified.

In one embodiment, the violation storage component 322 analyzes theviolation database to identify a physical layout feature having a highcorrelation to a violating activity. In one embodiment, the violationstorage component 322 analyzes the violation database to identify alocation within the work area having a high correlation to a violatingactivity. For example, the violation storage component 322 may determinethat a large number of violations are occurring at a specific locationin the work area or near a specific physical layout feature.Identification of such problem areas may allow for a reworking of thework area to be more efficient and/or reduce likelihood of violations.In one embodiment, the violation storage component 322 analyzes theviolation database to identify an individual having a high correlationto a violating activity. For example, the violation storage component322 may identify an individual having a high frequency for usingunwashed hands. The violation storage component 322 may also identifyprocedures that frequently occur and result in violations.

In one embodiment, a violation storage component 322 may identify workarea conditions which represent a high risk of violation. The violationstorage component 322 may generate a procedural rule that includesrequirements that correspond to the work area conditions related to thehigh risk of violation. The procedural component 308 may detect that therequirements of the procedural rule have been met and the notificationcomponent 310 may provide a warning within the work area that there is ahigh risk that a violation may soon occur. Individuals within the workarea may then be able to modify their actions to ensure that a violationdoes not take place. For example, the violation storage component 322may determine that when two types of objects come within a specificnon-contaminating proximity and when there are more than fourindividuals in the work area, there is a high risk that a violation willoccur. The two types of objects may include a sterile and a non-sterileobject. The procedural component 310 may detect these conditions andprovide a warning that a violation might occur based on the proximity ofthe objects and/or any other associated conditions.

The violation storage component 322 may assist in the identification ofwork area conditions that may cause a violation. Additionally, theviolation storage component 322 may be able to recognize warning signsthat a violation is about to occur and allow the notification component310 to provide a warning to prevent occurrence of the violation. Thus,not only can activities or contamination be monitored, but violationsand contaminations can actively be prevented by the activity monitoringcomponent 102.

Although the history component 316, activity storage component 318,tracking storage component 320, and violation storage component 322 andtheir associated databases or data files are described separately, theymay also be combined within a single component and/or database. In oneembodiment, for example, the history component 316, activity storagecomponent 318, tracking storage component 320, and violation storagecomponent 322 may all be included within the history component 316.Similarly, the history, activity database, tracking storage database,and/or violation database may all be included within one database.

The pest detection component 324 detects a pest within the work area.The pest detection component 324 may use the three-dimensional trackingsystem 104, sensors 106, cameras, or the like to detect a pest. Forexample, the pest detection component 324 may identify an object as apest based on one or more of object recognition in an image, a locationwithin the work area, how an object moves, or the like. For example,some pests tend to stick to corners or other specific locations within awork area. Similarly, pests may have characteristic manners in whichthey move through a work area. In one embodiment, the pest detectioncomponent 324 may identify an object as a pest when it appears to movewithout being manipulated by an individual. Some examples of pests thatmay be detected by the pest detection component 324 include rodents,insects, or the like. In one embodiment, the pest detection component324 may notify the procedural component 308 of the presence of a pest,and the procedural component 308 may determine that a violation hasoccurred based on the pest detection component detecting the pest.

The tracer detection component 326 detects a tracer material. Thethree-dimensional tracking system 104 may include the tracer detectioncomponent 326. In one embodiment, the tracer material includes amaterial that transfers to a contacting surface in response to contactwith the tracer material. For example, the tracer material may include adye, powder, or other material that transfers to a new object or surfacewhen contacted. Example tracer materials include a powder dye, a liquiddye, and/or a UV detectable dye.

The tracer trigger component 328 may trigger detectability of thetrigger material. For example, the tracer material may be difficult orimpossible to detect during normal activity within a work area. This mayallow a true test of procedures within a work area because individualsmay not be able to tell whether an object or material has a tracermaterial on it. The tracer trigger component 328 may triggerdetectability at a desired time. For example, the tracer triggercomponent 328 may trigger detectability at the end of a work areatraining session. In one embodiment, the tracer trigger component 328may include an ultra-violet (UV) source, and the tracer material mayinclude a UV detectable tracer. For example, the tracer material may beclear or colorless when UV light is not available and may then bevisible when a UV light source is directed towards the tracer material.

In one embodiment, the tracer material may be placed on one or moreobjects to be treated as contamination sources and the tracer detectioncomponent 326 may detect where the tracer material is. For example, overthe course of meal preparation in a kitchen, the tracking component 302may track the locations of the tracer material using the tracerdetection component 326 and may thereby determine what objects haveeffectively become contaminated through contact with the contaminationsource.

In one embodiment, the tracking component 302 determines that directcontact between a first surface and a second surface has occurred basedon the tracer detection component detecting the tracer material on boththe first surface and the second surface. Similarly, the trackingcomponent 302 may determine that sequential contact between a firstsurface, a second surface, and one or more intervening surfaces hasoccurred based on the tracer detection component 326 detecting thetracer material on the first surface and the second surface.

The procedural component 308 may determine whether an activity violatesthe one or more procedural rules based on the tracer detection component326 detecting the tracer material on an object of the one or moreobjects. Similarly, the activity identification component 304 maydetermine whether the activity violates the one or more procedural rulesbased on the tracer detection component 326 detecting the tracermaterial on the individual.

According to one embodiment, the tracer material may not be safe forhuman consumption and may be used solely for training purposes. Forexample, new employees of a restaurant may be provided with training inpreparing meals within a work area monitored by an contaminationmonitoring system 100 and mistakes or other details can be detected andpointed out. Thus, training can take place without risking contaminationof food that will be used by customers.

The embodiments, methods, systems, and functionality described above arediscussed largely in relation to activity and/or contaminationmonitoring in a kitchen work area. However, similar activity orcontamination monitoring within other types of work areas are alsopossible and fall within the scope of the present disclosure.

Additionally, considerable variation is possible even within kitchenwork areas. For example, kitchen work areas may include home kitchens,commercial, kitchens, or any other kitchen. The contamination monitoringsystem 100 may be used for contamination monitoring to detect foodcontamination, training purposes, general employee monitoring, customermonitoring, or the like. In one embodiment, an contamination monitoringsystem 100 may be installed in a work area in response to a legalviolation, food poisoning incident, or other problem at a commercialkitchen.

According to one embodiment, kitchens with employees having a mediumskill-levels in food preparation and handling may be most desirable foractivity monitoring. For example, high skill-level kitchens withextremely skilled or highly paid chefs may be less susceptible to foodhandling errors and low skill-level kitchens, such as at McDonalds®,often have extremely tightly programmed preparation procedures. Thus,middle skill-level kitchens, such as at Denny's®, where workers are notextremely skilled but also do not have tightly programmed preparationsprocedures, may be most desirable for monitoring.

FIG. 12 is a schematic flow chart diagram illustrating one embodiment ofa method 1200 for activity monitoring. The method 1200 may be performedby a contamination monitoring system 100 with any of the variationsdiscussed herein.

The method 1200 begins and a tracking component 302 tracks 1202 anindividual and one or more objects in a work area. The trackingcomponent tracks 1202 the location of the individual and objects using athree-dimensional tracking system 104. The position and/or orientationof the individual may also be tracked. In one embodiment, the trackingcomponent 302 tracks 1202 the locations of objects and the individual inrelation to each other and/or in relation to work area objects such ascounters, sinks, stoves, or the like. Similarly, the tracking component302 may track 1202 contact between objects, the individual, and/or thework area.

An activity identification component 304 identifies 1204 an activity ofthe individual. The activity identification component 1204 may identifythe activity based on movements, locations, or positions tracked 1202 bythe tracking component 302. In one embodiment, the activityidentification component 1204 may detect gestures performed by theindividual and determine 1204 an activity based on the detected gesture.The activity identification component 1204 may identify the activitybased, at least in part, on input from one or more sensors 106.

A procedural component 308 determines 1206 whether an activity violatesa procedural rule. The procedural component 308 determines 1206 whetherthe activity identified 1204 by the activity identification component1204 violates a procedural rule by comparing the activity andcircumstances within the work area to the procedural rule. For example,a procedural rule may define violation of a food handling procedure. Theprocedural component 308 may compare the activity, objects affected, orother circumstances to the food handling procedure. If the activitymatches the defined violation, the procedural component 308 determines1206 that the procedural rule has been violated.

The notification component 310 provides 1208 a notification of aviolation. For example, the notification component 310 may provide 1208the notification in response to the procedural component 308 determining1206 that a violation has occurred. The notification component 310 mayprovide the notification to a remote location, such as a remotecomputing system 110, and/or may provide a notification within the workarea itself. Notifications within the work area may include visible oraudio notifications. For example, visible notifications may be provided1208 using spotlights, laser pointers, display screens, indicator lightsmounted on objects, or the like. Audio notifications may be provided viaa speaker and/or sound card. In one embodiment, audio and visiblenotifications can be used in conjunction to increase awareness of errorsand/or provide information that teaches a user how to fix a problemand/or avoid violations in the future by following proper procedures.

FIG. 13 is a schematic flow chart diagram illustrating one embodiment ofa method 1300 for contamination monitoring. The method 1300 may beperformed by an contamination monitoring system 100 with any of thevariations discussed herein.

The method 1300 begins and a tracking component 302 tracks 1302 anindividual and one or more objects in a work area. The trackingcomponent 302 may track 1302 in a manner similar to that discussed abovein relation to FIG. 12.

The material identification component 306 identifies 1304 a material inthe work area. The material identification component 306 may identify1304 the material based on a color, pattern, shape, size, or texture ofan object or material. In one embodiment, the material identificationcomponent 306 receives an image of the material and analyzes the imageto identify 1304 the material. The material identification component 306may receive three-dimensional information for an object or material froma range finding device and determine a shape, size, texture, or the likebased on the three-dimensional information. The determined attributesmay then be used in identifying 1304 the material.

A procedural component 308 determines 1306 whether an object iscontaminated. The procedural component 308 may determine 1306 whether anobject is contaminated based on the movement of the object or themovement of another object, an individual or the like within the workarea. For example, proximity or contact between the object and othersurfaces may be tracked to determine if the object has contacted anunsanitary or contaminated surface. In one embodiment, the proceduralcomponent 308 may determine 1306 whether an object is contaminated basedon material types of the object or another object. Similarly, theprocedural component 308 may determine 1306 whether an object iscontaminated based on a sanitary or contamination status of a contactedsurface or object.

A notification component 310 provides 1308 a notification of acontamination. The notification component 310 may provide 1308 anotification remotely, in a visible manner, audible manner, or the like.The notification component 310 may provide 1308 a notification in anymanner disclosed in relation to providing 1208 a notification asdiscussed in relation to FIG. 12 or elsewhere within the presentdisclosure.

This disclosure has been made with reference to various exampleembodiments including the best mode. However, those skilled in the artwill recognize that changes and modifications may be made to theembodiments without departing from the scope of the present disclosure.For example, various operational steps, as well as components forcarrying out operational steps, may be implemented in alternate waysdepending upon the particular application or in consideration of anynumber of cost functions associated with the operation of the system,e.g., one or more of the steps may be deleted, modified, or combinedwith other steps.

Additionally, as will be appreciated by one of ordinary skill in theart, principles of the present disclosure may be reflected in a computerprogram product on a computer-readable storage medium havingcomputer-readable program code means embodied in the storage medium. Anytangible, non-transitory computer-readable storage medium may beutilized, including magnetic storage devices (hard disks, floppy disks,and the like), optical storage devices (CD-ROMs, DVDs, Blu-Ray discs,and the like), flash memory, and/or the like. These computer programinstructions may be loaded onto a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions that execute on thecomputer or other programmable data processing apparatus create a meansfor implementing the functions specified. These computer programinstructions may also be stored in a computer-readable memory that candirect a computer or other programmable data processing apparatus tofunction in a particular manner, such that the instructions stored inthe computer-readable memory produce an article of manufacture,including implementing means that implement the function specified. Thecomputer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer-implemented process, such that theinstructions that execute on the computer or other programmableapparatus provide steps for implementing the functions specified.

While the principles of this disclosure have been shown in variousembodiments, many modifications of structure, arrangements, proportions,elements, materials, and components, which are particularly adapted fora specific environment and operating requirements, may be used withoutdeparting from the principles and scope of this disclosure. These andother changes or modifications are intended to be included within thescope of the present disclosure.

The foregoing specification has been described with reference to variousembodiments. However, one of ordinary skill in the art will appreciatethat various modifications and changes can be made without departingfrom the scope of the present disclosure. Accordingly, this disclosureis to be regarded in an illustrative rather than a restrictive sense,and all such modifications are intended to be included within the scopethereof. Likewise, benefits, other advantages, and solutions to problemshave been described above with regard to various embodiments. However,benefits, advantages, solutions to problems, and any element(s) that maycause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, a required, or anessential feature or element. As used herein, the terms “comprises,”“comprising,” and any other variation thereof, are intended to cover anon-exclusive inclusion, such that a process, a method, an article, oran apparatus that comprises a list of elements does not include onlythose elements but may include other elements not expressly listed orinherent to such process, method, system, article, or apparatus. Also,as used herein, the terms “coupled,” “coupling,” and any other variationthereof are intended to cover a physical connection, an electricalconnection, a magnetic connection, an optical connection, acommunicative connection, a functional connection, and/or any otherconnection.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

What is claimed is:
 1. A computer-readable storage medium comprisingprogram code for causing one or more processors to perform a method forcontamination monitoring, the method comprising: tracking an individualand one or more objects in a work area using a three-dimensionaltracking system; identifying a material of the one or more objects basedon a captured image; determining that an object of the one or moreobjects is contaminated based on tracked locations of the one or moreobjects and the individual; and providing a notification of thecontamination, objects; and providing a notification of a violation.2.-5. (canceled)
 6. The computer-readable storage medium of claim 1,wherein identifying the material is based on a color of the material inthe captured image.
 7. The computer-readable storage medium of claim 1,wherein identifying the material is based on a pattern on the materialin the captured image.
 8. The computer-readable storage medium of claim1, wherein identifying the material is based on a texture of thematerial. 9.-10. (canceled)
 11. The computer-readable storage medium ofclaim 1, wherein identifying the material is based on a shape of thematerial. 12.-13. (canceled)
 14. The computer-readable storage medium ofclaim 1, wherein identifying the material comprises identifying thematerial as raw meat. 15.-18. (canceled)
 19. The computer-readablestorage medium of claim 1, wherein identifying the material comprisesidentifying the material as a vegetable.
 20. The computer-readablestorage medium of claim 1, wherein identifying the material comprisesidentifying the material as unsanitary.
 21. The computer-readablestorage medium of claim 20, wherein identifying as unsanitary comprisesidentifying the material as having a high risk of contaminating throughproximity one or more of another object of the one or more objects andthe individual.
 22. The computer-readable storage medium of claim 20,wherein identifying as unsanitary comprises identifying raw meat asunsanitary.
 23. The computer-readable storage medium of claim 20,wherein identifying as unsanitary comprises identifying raw animalproducts as unsanitary.
 24. The computer-readable storage medium ofclaim 1, wherein identifying the material comprises identifying thematerial as sanitary.
 25. The computer-readable storage medium of claim24, wherein identifying as sanitary comprises identifying the materialas having a low risk of contaminating through proximity with an objectof the one or more objects and the individual.
 26. The computer-readablestorage medium of claim 24, wherein identifying as sanitary comprisesidentifying washed vegetables as sanitary.
 27. The computer-readablestorage medium of claim 24, wherein identifying as sanitary comprisesidentifying baked goods as sanitary.
 28. (canceled)
 29. Thecomputer-readable storage medium of claim 1, wherein determining thatthe object is contaminated is based on the material of the object. 30.The computer-readable storage medium of claim 1, wherein determiningthat the object is contaminated comprises determining that the objecthas contacted a contamination source, the contamination sourcecomprising a material that doesn't belong with the object.
 31. Thecomputer-readable storage medium of claim 30, wherein the contaminationsource comprises another object of the one or more objects.
 32. Thecomputer-readable storage medium of claim 30, wherein the contaminationsource comprises the individual.
 33. The computer-readable storagemedium of claim 30, wherein the contamination source comprises a workarea object. 34.-112. (canceled)
 113. The computer-readable storagemedium of claim 1, wherein tracking an individual and one or moreobjects in a work area comprises tracking the individual and the one ormore objects in a kitchen. 114.-142. (canceled)
 143. Thecomputer-readable storage medium of claim 1, wherein tracking using athree-dimensional tracking system comprises tracking using a rangeimaging system. 145.-153. (canceled)
 154. The computer-readable storagemedium of claim 1, wherein tracking using a three-dimensional trackingsystem comprises tracking using one or more cameras and a range finder.155.-169. (canceled)
 170. The computer-readable storage medium of claim1, the method further comprising identifying the individual. 171.-284.(canceled)
 285. A system for contamination monitoring, the systemcomprising: a tracking component to track an individual and one or moreobjects in a work area using a three-dimensional tracking system; amaterial identification component to identify a material of the one ormore objects based on a captured image; a procedural component todetermine that an object of the one or more objects is contaminatedbased on tracked locations of the one or more objects and theindividual; and a notification component to provide a notification ofthe contamination. 286.-328. (canceled)
 329. The system of claim 285,wherein the procedural component determines that the object iscontaminated based on determining that there is high risk that theobject comprises material that doesn't belong with the object. 330.-334.(canceled)
 335. The system of claim 285, wherein the proceduralcomponent determines that the object is contaminated based on sequentialcontact between the object, one or more intermediate objects, and acontamination source.
 336. The system of claim 285, wherein theprocedural component determines that the object is contaminated based ona location of the object within a logical area of the work area
 337. Thesystem of claim 336, wherein the procedural component determines thatthe object is located in a meat handling area.
 338. The system of claim336, wherein the procedural component determines that the object islocated in a hand washing area. 339.-340. (canceled)
 341. The system ofclaim 285, further comprising a history component to maintain a historyfor one or more of the one or more objects, the individual, and portionsof the work area.
 342. The system of claim 341, wherein the proceduralcomponent determines that the object is contaminated based on thehistory. 343.-346. (canceled)
 347. The system of claim 285, furthercomprising a status component to maintain a contamination status for oneor more of the one or more objects, the individual, and a portion of thework area. 348.-350. (canceled)
 351. The system of claim 285, whereinthe notification component providing the notification comprises thenotification component indicating in the work area a contaminationstatus of one or more of an object of the one or more objects, theindividual, and a portion of the work area.
 352. The system of claim351, wherein indicating comprises the notification component visuallyindicating the contamination status. 353.-356. (canceled)
 357. Thesystem of claim 351, wherein indicating comprises the notificationcomponent audibly playing a sound indicating the contamination status ofthe object. 358.-412. (canceled)
 413. The system of claim 285, whereinthe tracking component tracking comprises tracking an individual and oneor more objects in a work area logically divided into two or morelogical areas.
 414. The system of claim 413, wherein the two or morelogical areas comprise a raw meat area. 415.-569. (canceled)
 570. Amethod for contamination monitoring, the method comprising: tracking anindividual and one or more objects in a work area using athree-dimensional tracking system; identifying a material of the one ormore objects based on a captured image; determining that an object ofthe one or more objects is contaminated based on tracked locations ofthe one or more objects and the individual; and providing a notificationof the contamination.